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STUDY  REPORT  ABSTRACT:  This  report  examines  new  training  concepts  developed 

throughout  DOD  over  the  past  decade.  The  concepts  that  show  promise  for 
reducing  life  cycle  costs  are  considered  for  trade-offs  with  hardware  devel- 
opments. The  process  of  trade-off  considerations  is  treated  with  a marginal 
cost-marginal  benefit  analysis  (put  the  investment  where  it  provides  the 
biggest  return).  The  writer  then  conducts  a sensitivity  analysis  on  parameters 
affected  by  training  using  a computer  model  to  determine  a trend  in  life  cycle 
costs/savings.  The  writer  concludes  that  many  benefits  can  be  derived  by 
increasing  the  share  of  the  investment  in  the  training  subsystem  although  it 
may  be  at  the  cost  of  the  hardware  subsystem  in  some  cases.  These  investments 
appear  to  be  best  placed  in  training  and  technical  documentation  or  in  job 
performance  aids.  He  concludes  that  not  only  should  this  type  investment 
reduce  the  life  cycle  cost  but  also  provide  job  enrichment,  higher  operational 
availability,  fewer  maintenance  personnel  requirements,  fewer  training  course 
requirements,  increased  systems  effectiveness,  and  other  savings. 

The  implication  of  the  study  is  that  Project  Management  personnel  concentrate 
on  the  hardware  aspects  of  a developing  system  while  investments  in  training 
developments  provide  a greater  return  on  the  later  investment  and  would  yield 
greater  user  satisfaction. 
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EXECUTIVE  SUMMARY 


This  study  project  examines  the  benefits  of  recent  Training  Develop- 
ments which  are  the  results  of  integrating  technical  documentation  and 
training  techniques  into  a composite  package.  It  also  examines  the  results 
of  Training  Development  studies  conducted  in  all  three  US  Services.  It 
includes  insights  into  the  problems  of  achieving  expected  operational 
availability,  maintainability,  and  lower  Life  Cycle  Costs  for  new  systems, 
that  is  the  total  cost  related  to  a system  over  its  life  such  as  'esearch 
and  development,  procurement,  operations  and  support,  and  salvage.  The 
maintenance  documentation  concepts  are  examined  illustrating  the  inter- 
relationships of  technical  and  training  documentation  with  systems  main- 
tainability. The  implications  of  using  documentation  concepts  tailored 
for  specific  equipment  types  were  investigated  with  the  expected  results 
recorded  herein.  Representative  findings  from  the  studies,  tests,  and 
experiments  examined  were  used  as  computer  data  changes  to  a life  cycle 
cost  model  with  the  computed  results  included  in  this  report.  These  results 
were  examined  in  light  of  recent  DoD  guidance  to  reduce  Life  Cycle  Costs. 

It  is  concluded  that  the  proper  selection  of  promising  training  development 
concepts  such  as  technical  and  training  documentation  will  reduce  life 
cycle  costs  and  increase  operational  availability.  Other  more  specific 
expectations  are  reduced  personnel  requirements,  reductions  in  training 
time,  decreased  mean  time  to  repair,  fewer  job  specialties,  reduction  in 
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spare  parts  usage,  greater  user  satisfaction,  and  a more  usable  support 
package  for  foreign  military  sales. 

It  is  further  concluded  that  an  overall  systems  effectiveness  approach 
is  required  to  give  sufficient  emphasis  to  training  and  support  and  insure 
project  managers  plan  and  contractually  support  the  selection  and  develop- 
ment of  the  most  effective  materials. 

It  is  also  concluded  that  revision  of  existing  technical  and  training 
a documentation  may  be  a viable  alternative  to  upgrading  system  performance 

and  may  be  more  cost  effective  than  the  customary  hardware  modifications 
of  Product  Improvement  Programs  (PIP). 

The  implication  of  the  study  is  that  present  practices  emphasize 
inherent  hardware  reliability  and  availability  values  while  ignoring 
enhancement  through  improved  training  developments,  a relatively  simple 
means  of  receiving  greater  payoffs  and  increased  user  satisfaction  while 
reducing  life  cycle  costs. 
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PREFACE 

This  study  was  undertaken  because  the  author,  while  a member  of  the 
Army  Combat  Development  efforts  in  Training  and  Doctrine  Command  (TRADDC) 
over  the  past  six  years,  has  seen  increased  adversary  relations  develop 
in  the  Army  between  the  developer  and  user.  During  that  time,  while 
working  as  an  Operations  Research  Systems  Analyst,  he  observed  that  the 
system  values  achieved  by  the  developer  in  response  to  the  requirements 
document  (such  as  operational  reliability,  availability,  and  maintain- 
ability) were  in  fact  closer  to  inherent  (or  hardware)  values  and  were 
never  or  rarely  the  values  achieved  either  in  user  tests  during  develop- 
ment or  after  fielding.  It  was  apparent  to  the  author  that  failure  to 
consider  the  man  interface  was  a primary  reason  for  the' degraded  perfor- 
mance. ' 

The  author  was  instrumental  in  developing  much  of  the  Army  Systems 
Acquisition  user  test  policy  from  1974  - 1976.  He  also  attended  during 
that  period  numerous  meetings  between  developer  and  user  commands  where 
test  failures,  failure  definitions  and  scoring  criteria,  training  and 
logistics  shortcomings  and  overall  systems  effectiveness  were  related  and 
discussed  for  resolution.  With  this  background-' and  the  knowledge  that  many 
studies,  tests,  and  experiments  related  to  training  and  technical  documen- 
tation developments  had  been  conducted  with  cost  saving  recommendations, 

t 

this  task  was  undertaken.  The  test  and  study  results  also  hinted  at  a 
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method  to  improve  the  military  readiness  levels  with  a reduction  in  life 
cycle  costs  while  increasing  user  satisfaction  and  equipment  utility/ 
desirability. 
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SECTION  I 


INTRODUCTION 

The  job  of  maintaining  a military  force  in  a high  state  of  readiness 
falls  on  the  military  commander.  On  the  modern  battlefield  he  is  going  to 
be  equipped  with  sophisticated  weapons  and  support  equipment  more  complex 
than  ever  before.  He  has  a major  task  of  maintaining  those  weapons  in  a 
high  state  of  readiness  and  to  employ  them  in  their  mission  whenever 
required  to  enter  the  battle.  However,  for  years  reports  from  users 
throughout  the  world  have  expressed  concern  and  great  difficulty  in  main- 
taining the  readiness  conditions  required.  Many  factors  have  been  given 
as  contributing  causes  such  as  lack  of  repair  parts,  high  equipment  failure 
rates,  long  times  to  repair,  and  inadequately  trained  maintenance  personnel. 
Our  servicemen  and  the  country  deserve  equipment  that  once  employed  in 
battle  can  be  expected  to  remain  operational  for  the  duration  of  the  mission. 
Any  system  found  to  be  non-operable  should  be  expected  to  be  brought  back 
to  an  operable  condition  in  the  minimum  amount  of  time. 

In  a statement  to  the  94th  Congress  Second  Session  3 ^eb*1976  concern- 
ing DoD  Research,  Development,  Test  and  Evaluation  Program,  Dr.  Malcolm 
Currie  stated: 

"At  present,  while  the  development  and  procurement  costs 
of  DoD  systems  and  equipment  are  well  identified,  our 
understanding  of  the  real  life  cycle  costs  are  sketchy 
at  best,  because  of  the  way  DoD  operation  and  support 
costs  are  reported. 
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Because  knowledge  of  total  Life  Cycle  Costs  is  vital 
to  decision-making,  OSD  has  expanded  its  effort  to 
improve  visibility  of  operating  and  support  costs, 
so  that  all  cost  drivers  can  be  identified  and  timely 
efforts  to  reduce  them  initiated " 

"Since  reliability,  maintainability,  and  dvect 
personnel  costs  influence  operating  and  support  costs, 
the  DSARC  has  been  imposing  more  stringent  require- 
ments for  reporting  actual  field  reliability  and  main- 
tainability achievements  and  related  support  costs 
throughout  the  life  of  each  program.  This  action  will 
focus  increased  management  attention  on  these  elements 
of  life  cycle  costs...." 

Lieutenant  General  George  Sammet,  Jr.,  Deputy  Commandin'  General  for 

Materiel  Development,  Army  Materiel  Development  and  Readiness  Command, 

speaking  at  the  American  Defense  Preparedness  Association  Tank  Automotive 

Division  Meeting  in  Monterey,  California,  17  Nov  76  stated: 

"...Right  along  with  reliability  goes  maintainability. 

I don't  care  how  reliable  a vehicle  you  put  in  the 
field,  sooner  or  later  it's  going  to  break  down  and 
need  fixing.  The  easier  and  faster  it  is  to  do  that 
job,  the  more  ready  we  will  be. 

Along  with  good  maintenance  is  good  technical  data.  If 
you  are  going  to  tune  up  your  Ford  you  can't  use  a 
Jaguar  manual  to  tell  you  how  to  do  it.  Using  some  of 
our  manuals  in  thepast  was  about  like  using  a Jaguar 
manual  to  fix  a Ford.  Our  old  manuals  were  difficult 
to  read,  or  there  were  no  illustrations,  or  the  illus- 
trations were  all  wrong  - or  at  least  wrongly  placed  in 
the  manual.  Some  are  still  that  way.  But  we  are 
correcting  this  situation.  We  have  to.  if  we  are  truly 
going  to  have  equipment  readiness. 

If  a soldier  is  going  to  maintain  his  equipment,  the 
soldier  is  going  to  have  to  be  able  to  troubleshoot  it. 

To  troubleshoot  it  he's  going  to  need  test  and  support 
equipment.  Too  many  times  in  the  past  we've  fielded  a 
new  piece  of  equipment  long  before  the  less  glamorous 
test  and  support  equipment  was  ready." 
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Later  he  also  stated: 

"You  can't  put  reliability  in  a vehicle  after  it's  in 
the  field.  ...Reliability  has  to  be  built  into  a 
vehicle,  and  its  not  going  to  get  there  by  accident. 

The  same  goes  for  Tech  (technical)  Manuals.  They  don't 
write  themselves,  yet  manuals  are  as  much  a part  of 
readiness  as  the  mechanics  tool  kit " 

Also,  while  discussing  support  cost  and  Design  to  Unit  Production 
Cost  (DTUPC)  he  stated: 

"Our  critics  argue  that  we  should  have  been  talking 
DTUPC  and  life  cycle  costing  all  the  time.  Since  the 
life  cycle  cost  is  a lot  tougher  number  to  predict,  we 
did  push  it  into  the  background.  Well,  its  time  has 
come  - as  they  say  on  TV  - to  get  the  whole  package 
under  a life  cycle  cost  figure." 

Over  the  past  ten  years,  DoD  has  put  much  emphasis  on  improving  the 
maintenance  of  the  hardware  by  emphasizing  reliability  and  maintainability 
as  design  criteria  and  the  development  of  automatic  and  built-in  test 
equipment.  Very  little  attention,  however,  has  gone  into  improving  the 
maintenance  information.  The  conventional  technical  manual  system  has 
existed  without  change  for  decades.  Some  gradual  improvement  has  occurred 
but  there  has  been  no  innovative  thrust  to  improve  the  effectiveness  of 
the  people  portion  of  our  maintenance  system.  Some  examples  of  these 
technical  manual  problems  and  possible  solutions  are  at  Figures  1 1 1 - 1 and 
1 1 1 - 2 . Also,  as  stated  by  Dr.  Currie  later  in  his  referenced  presenta- 
tion, approximately  25%  of  all  military  personnel  are  employed  in  full 
time  maintenance  service.  This  does  not  include  operators  of  trucks,  tanks, 
guns,  etc.  If  we  consider  the  individual  cost  at  $20,000  per  man  year  (a 
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low  estimate)  this  represents  $10  billion  per  year  (this  figure  varies 
with  total  DoD  strength, currently  2.1M).  If  we  then  inject  that  one- 
fourth  of  these  men  are  new  to  the  service  each  year  and  that  training 
costs  about  $1,000  per  week,  with  the  service  school  technical  course 
averaging  20  weeks  to  get  them  ready  for  work  (with  no  useful  output 
during  training)  that  adds  $2.4  billion  per  year^'^.  The  DoD  main- 
tenance personnel  support  is  easily  in  excess  of  $12.4  billion  per  year. 

The  primary  purpose  for  stating  these  facts  is  to  put  emphasis  on  the 

publications,  training  and  personnel  systems  - three  of  the  many  dis- 
(3) 

integrated'  ' elements,  reference  figure  1-1 , of  systems  under  develop- 
ment, each  of  which  should  integrate  and  improve  system  effectiveness  by 
better  training  developments.  Within  the  DoD  community  many  experiments 
and  demonstrations  have  shown  that  improvements  to  technical  manuals  and 
other  new  ways  of  presenting  information  may  enable  technicians  to  perform 
faster  and  more  accurately,  and  with  less  training.  These  techniques  have 
the  potential  of  reducing  the  cost  of  maintenance  and  maintenance  support 
(training)  by  25-50%  and  at  the  same  time  improve  the  equipment  availability. 

Two  studies  conducted  for  DoD  elements  in  recent  years  were  relied  on 

(1  2) 

heavily  for  this  paper'  ’ ' . Each  of  them  reviewed  and  reported  the  many 
techniques  and  concepts  used  in  improving  training  and  maintenance.  This 
paper  will  present  the  results  of  many  of  these  examinations,  discuss  the 
applications  and  recommend  several  steps  that  DoD  should  take  in  implemen- 
ting the  results. 
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PURPOSE  OF  THE  STUDY  PROJECT 
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It  is  the  purpose  of  this  study  project  to  examine  the  results  of 
studies  and  experiments  concerning  new  training  development  concepts 
(Section  III)  and  to  consider  the  influence  on  System  Life  Cycle  Costs 
when  applying  these  new  concepts. 

SPECIAL  GOALS  OF  THIS  PROJECT 

The  goal  of  this  project  is  to  answer  the  following  question:  Can 

training  and  technical  documentation  developments  be  incorporated  into 
trade-off  considerations  with  hardware  developments  for  achieving  lower 
life  cycle  costs  and  if  so,  how? 

LIMITATIONS  OF  THE  STUDY 

The  limitations  of  time  and  resource  material  were  the  primary 
limiting  factors.  During  the  data  gathering  phase  it  was  desirable  but 
not  practical  due  to  limited  time  to  visit  existing  program  management 
offices  to  develop  more  information  on  problems  of  implementation  and 
interface.  Also  due  to  time  it  was  not  possible  to  obtain  data  from  the 
many  industry  organizations  or  professional  societies  that  have  worked  on 
this  subject. 

SIGNIFICANCE  OF  THE  STUDY 

Many  Department  of  Defense  Directives  and  Instructions  have  been  pub- 
lished stating  Life  Cycle  Cost  will  be  used  as  a constraint  in  System 
Acquisition.  Other  documents  have  stated  the  importance  of  user  involvement. 
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However,  the  primary  emphasis  of  program  managers  and  the  system  acquisi- 
tion process  continues  to  be  hardware  development  and  acquisition.  While 
it  is  obvious  that  the  primary  element  of  the  system  initially  is  the 


hardware,  and  the  early  concern  is  in  hardware  development,  it  is  not  at 
all  clear  to  this  writer  why  the  'hardware  fascination1  continues  through- 
out the  acquisition  process  at  the  expense  of  the  other  system  elements. 

If  the  many  results  of  this  study  are  recognized  by  DoD  policy  makers  as 
worthwhile,  the  'hardware  acquisition  process'  will  become  the  'system 
acquisition  process'  in  fact  and  not  just  in  print.  The  benefits  of  such 
a change  are  obvious  as  hardware  is  only  one  of  the  many  subsystems  of 
the  total  system. 

ORGANIZATION  OF  THE  STUDY 

The  study  is  organized  into  sections.  The  first  introduces  the  study 
with  its  purpose,  goals,  limitations,  significance,  and  organization.  The 
second  section  discusses  the  meaning  of  training  development  and  other 
related  support  terms.  The  third  section  presents  the  results  of  many 
studies,  tests,  and  experiments  that  have  been  undertaken  to  examine  new 
concepts  of  training  and  technical  documentation  developments.  The  fourth 
section  translates  the  findings  of  section  three  into  usable  information 
for  decision  making.  Trade  off  analyses  are  considered  by  modeling  for 
Life  Cycle  Costs,  considering  marginal  cost  and  marginal  profit  analysis, 
and  examining  cost  drivers.  Section  five  presents  conclusions,  recommen- 
dations and  implications  of  the  study  project. 


7 


i 


SECTION  II 

TRAINING  DEVELOPMENTS  AND  OTHER 
RELATED  SUPPORT  DEFINITIONS 

The  purpose  of  this  section  is  to  examine  the  meaning  of  training 
developments  and  related  terms  such  as  Job  Performance  Aids,  technical 
data,  and  maintainability  as  used  in  this  study. 

TRAINING  DEVELOPMENT 

f 4 • 33 1 

Training  development  as  evolving  throughout  DoD'  ' ' integrates 

technical  documentation  and  training  into  a composite  package  which  pro- 
vides a reference  state  of  all  information  needed  to  operate  and  maintain 
an  equipment  system.  The  technical  documentation  contains  fully  detailed, 
illustrated  instructions  which  enable  novice  technicians  to  perform  complex 
tasks  with  little  or  no  training.  The  training  materials  consist  of  self 
study  lessons  which  teach  the  preliminary  skills  necessary  to  use  the 
documentation  in  operating  and  maintaining  the  equipment.  For  either 
existing  or  evolving  tasks,  a developmental  program  to  accomplish  these 
new  goals  is  to  take  a macrocosmic  view  of  the  system/problem  and  bring 
together  the  Job  Performance  Aids  (JPA)  approach  to  data  presentation  and 
training. 

JOB  PERFORMANCE  AIDS 

Job  Performance  Aids  may  be  booklets  or  viewing  devices  which  present 
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text.  They  present  the  soldier  with  technical  data  in  an  illustrated, 
easy-to-read,  step-by-step  format  supported  by  a training  package  which 
integrates  the  necessary  material  to  teach  critical  tasks,  frequently 
recurring  tasks,  and  procedures  relative  to  troubleshooting,  safety,  and 
emergencies.  Key  features  of  the  new  training  development  concepts  are 
that<4:34>: 

o Development  is  based  on  an  analysis  of  job  and  task  requirements. 

o Material  is  developed  with  the  target  audience  and  work  conditions 
in  mind. 

o Technical  data  is  presented  in  job  sequence  and  organized  for 
accessibility  (figure  1 1 -1 ) . 

o Training  and  Technical  Documentation  are  integrated. 

o Technical  manuals  and  training  packages  are  verified  by  users 
under  realistic  conditions  prior  to  acceptance. 

Man's  limitations  and  capabilities  must  be  considered  early  in  the 
hardware  design  and  then  personnel  must  be  trained  to  the  skill  level 
required  to  operate,  maintain,  and  support  systems  and  equipment  for 
ultimate  effectiveness.  They  can  do  this  only  with  proper  technical  data. 

TECHNICAL  DATA 

Technical  data  provides  the  link  between  the  man  and  the  machine,  the 
maintenance  technician  and  the  part,  the  driver  and  his  tank.  Technical 
data  are  written  instructions  such  as  drawings;  operating,  maintenance, 
and  modification  manuals,  specifications,  inspection  test,  and  calibration 
procedures;  and  computer  programs  required  to  guide  people  performing 
operations  and  support  tasks.  An  example  of  technical  data  are  technical 
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WHEN  IT  SAYS... 
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2.  DISCONNECT  AID  SELECTOD  VALVE 
CABLE  (G.  FIGURE  2-2)  FROM  AIR 
SELECTOR  VALVE  PIVOT  SHAFT  LEVER 
ON  TOP  OF  INLET  REDUCING  DUCT  BY 
REMOVING  COTTFR  PIN,  WASHER,  AND 
FLAT  HEAD  PIN  SECURING  CABLE  FORK 
END  TO  LEVER. 

AND  SHOWS  ... 


orders  giving  instructions  on  how  to  repair  equipment.  Delivery  of  tech- 
nical data  at  the  same  time  the  corresponding  hardware  is  delivered  is  an 
important  objective  of  the  Project  Manager.  It  is  difficult  and  impractical 
to  maintain  or  operate  much  of  the  complex  equipment  now  being  produced 
without  clear  and  simple  written  guidance.  Thus  the  scheduling  of  tech- 
nical data  developments  and  distribution  must  mesh  with  equipment  produc- 
tion schedules. 

MAINTAINABILITY 

Maintainability  is  a design  characteristic  which  is  expressed  as  the 
probability  that  an  item  will  be  retained  in,  or  restored  to,  a specified 
condition  within  a given  time  period  when  maintenance  action  is  performed 
according  to  the  prescribed  procedures.  A parameter  often  used  as  an 
expression  of  maintainability  is  mean  time  to  repair  (MTTR).  Maintain- 
ability is  a quantitative  characteristic  that  can  be  predicted  in  design, 
controlled  in  production,  measured  by  test,  and  maintained  in  the  field. 

A Reminder 

Maintainability  is  the  ability  of  the  item  to  be  maintained  whereas 
mai ntenance  is  a series  of  actions  to  restore  or  retain  the  item  in  the 
specified  condition. 

The  maintainability  characteristic  of  hardware  dominates  the  number 
and  types  of  trained  personnel,  tools,  equipment,  spare  parts,  and  technical 
data  for  support  of  the  equipment  in  an  operational  environment. 
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Maintenance,  operations,  and  personnel  costs  as  shown  on  the  following 
graph^  account  for  an  estimated  50%  of  a systems  total  life  cycle  costs 
and  are  projected  to  account  for  this  portion  in  the  future  (reference 
figure  1 1 -2). 
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SECTION  III 


RESULTS  OF  STUDIES  EXAMINING  NEW 
CONCEPTS  IN  TRAINING  AND  DOCUMENTATION 

The  purpose  of  this  section  is  to  present  the  results  of  the  many 

studies,  tests,  and  experiments  conducted  to  develop  improvements  in  the 

military  training  and  technical  documentation. 

There  have  been  approximately  eighty  ‘new  concepts'  such  as  the 

'before'  and  'after'  extracts  on  the  following  pages,  designed  or  developed 

to  make  improvements  on  the  performance  of  the  personnel,  primarily  through 
(2-12) 

improved  manuals  ' . The  studies  examining  the  change  of  content  and 

technical  information  format  show  some  surprising  things.  For  example, 

25  research  reports  on  some  30  new  concepts  report  improvements  in  job 
performance  or  reduction  of  training  time  when  manuals  incorporating  the 
new  concepts  were  used.  These  study  reports  were  examined  with  the 
following  two  questions  in  mind: 

1.  How  are  the  research  results  converted  into  lowering  the  cost  of 
ownership,  and 

2.  What  are  the  fundamental  processes  that  make  these  new  concepts 
so  effective? 

The  examinations  revealed  that  improvements  must  come  from  one  of  these: 

1.  Reduced  training  time. 

2.  Reduced  number  of  personnel  required  to  do  the  job.  (Increased 
proficiency  of  individual  men.) 

3.  Reduced  errors  (reduction  of  false  removals  improved  by  increased 
proficiency) . 
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Figure  III  - 2 

Example  of  Change  to  a Training  Manual 
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The  conditions  for  most  studies  were  essentially  those  that  exist  on 
the  iob.  In  some  cases  the  maintenance  that  was  performed  occurred  in 
response  to  actual  schedules  and  actual  malfunctions.  In  other  cases  time 
was  'compressed'  and  maintenance  personnel  responded  to  equipment  malfunc- 


tions that  were  introduced  by  the  experimenters.  In  any  case  the  test 

malfunctions  were  not  selected  as  the  ones  that  special  manuals/techniques 

were  designed  to  find.  The  special  manuals  were  designed  to  treat  any 

problem  in  maintenance  of  the  hardware  system.  Thousands  of  maintenance 

problems  in  perhaps  a hundred  different  hardware  systems  are  represented 

in  all  of  the  research  tests  surveyed.  The  studies,  tests,  and  experiments 

were  conducted  in  all  three  services  and  the  Coast  Guard  over  a period  of 

15  years.  The  following  reduction  estimates  were  made  by  both  Shriver^'^ 
(?•?) 

and  Rowan'  ' , two  years  apart  looking  at  similar  but  not  identical  data 

(Table  III-l )(12:154); 

o Reduction  in  training  of  75% 
o Reduction  in  work  force  of  15% 
o Reduction  in  false  removals  of  15% 

The  results  clearly  indicate  that  improvements  in  the  maintenance  personnel 

effectiveness  are  possible  with  potential  of  high  cost  savings. 

(2-2l 

Rowan's  findings  also  included'  ’ ' that  these  reductions  could  be 
achieved  at  a cost  within  20-25"  of  the  cost  now  being  expended  for  conven- 
tional technical  manuals.  If  only  one  study  revealed  significant  results, 
the  reader  may  be  tempted  to  say  it  was  a coincidence.  But  there  have  been 
more  than  twenty-five  studies  in  DoD  and  industry.  They  all  point  in  the 


same  direction.  Before  going  on  the  reader  should  be  aware  of  the  tradi- 
tional assumptions  about  recording  or  passing  technical  information.  They 


are: 

1.  Technical  training  is  needed  to  provide  the  general  theoretical 
thesis  of  the  subject  matter  (electronics,  mechanical,  hydraulics, 
etc. ) . 

2.  Technical  manuals  shouid  describe  the  specifics  of  the  system 
to  be  maintained. 

3.  The  maintenance  man  can  and  will  combine  1 and  2 on  the  job  to 
determine  what  should  be  done  in  each  situation  as  it  arises. 

The  assumptions  common  to  the  new  concepts  are  that  experts  can  analyze 
the  equipment,  determine  what  should  be  done  to  it  in  every  possible  situa- 
tion, and  record  this  in  technical  manuals  so  that  even  a novice  can  use 
it  to  perform  the  task  correctly.  Although  the  documentation  technique 
for  troubleshooting  tasks  was  slightly  different  than  those  for  non-trouble- 
shooting,  the  trend  and  results  were  the  same.  Using  the  new  kinds  of 
documentation  (Figures  111-1,2),  personnel  with  half  as  much  training  as 
groups  using  standard  technical  manuals,  were  able  to  correct  three  times 
as  many  malfunctions  per  unit  time  in  a major  radar  system  All  the 

concepts  are  directed  at  telling  the  man  what  to  do  instead  of  teaching  him 
about  the  system  details  and  training  in  what  is  basically  design  engineer- 
ing information.  It  emphasizes  training  and  not  teaching.  The  new  types 
of  documentations  provide  a more  intelligible  product.  Intelligibility 
is  enhanced  by: 

1.  Readability  - A noun  and  verb  list  and  limit  on  sentence  length 
that  limits  the  manual  to  a fifth  grade  reading  level. 

18 


2.  Organization  - A process  of  arranging  the  material  in  the  book 
to  reduce  the  need  to  shift  from  one  part  of  the  book  to  another 
while  performing  a continuing  procedure. 

3.  Graphics/Text  Correlations  - This  involves  using  pictures  and 
graphic  techniques  that  allow  the  novice  to  make  a 'match' 
between  what  he  sees  in  the  manual  and  what  he  sees  on  the 
equipment. 

While  one  would  assume  that  the  current  maintenance  documentation  will 
do  these  things,  it  rarely  does.  In  analyzing  the  maintenance  actions 
requiring  for  the  doppler  radar  system  of  the  C-141 , the  Air  Force  found 
that  the  isolation  and  repair  of  one  malfunction  required  reference  to 
165  pages  of  8 documents^ ' ^ . 

(2*12) 

The  following  table  of  experiments  was  extracted'  ' ' showing  the 

concepts  used  and  results  achieved. 

In  all  of  the  experiments  and  field  tests,  inexperienced  technicians 
performed  better  with  fully  procedural i zed  aids  than  with  conventional 
documentation.  Often,  particularly  when  troubleshooting,  the  inexperienced 
technicians,  even  those  who  had  attended  the  prescribed  school,  were  unable 
to  perform  at  all  using  conventional  manuals.  However,  they  were  able  to 
perform  with  minimal  errors  using  the  aids  and  their  time  to  repair 
approached  the  time  experienced  technicians  required  when  the  latter  used 
either  conventional  or  experimental  documentation. 

To  summarize  some  key  findings  of  the  studies: 

o Repair  time  was  reduced  by  up  to  33  percent,  with  80  percent  fewer 
errors  (BFIC)  (2:13) 

o As  much  as  forty-two  percent  more  malfunctions  were  found  in  41 
percent  less  time  (MAINTRAIN)  (2:12) 
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o Diagnosis  time  fell  by  as  much  as  67  percent  (British  Algorithm) 
(2:13) 

o Training  time  was  reduced  as  much  as  60  percent  for  trouble- 
shooting tasks  and  up  to  100  percent  for  non-troubleshooting.  (1:35) 

o Inexperienced  technicians  using  Job  Performance  Aids  performed 
better  than  experienced  technicians  using  traditional-type 
technical  manuals  (Non-Troubleshooting  JPAs)  (2:13). 


TABLE  III-1 

Experimental  Comparisons  of  Innovative  Job  Aids 
Versus  Conventional  Documentation 
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VNAF  experienced  technicians 
performed  equally  well  with 
all  aids,  apprentices  did 
better  with  JPA,  outperforming 
experienced  technicians. 
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SECTION  IV 


CONSIDERATIONS  OF  APPLYING  THESE  CONCEPTS 
TO  REDUCE  LIFE  CYCLE  COSTS 

The  purpose  of  this  section  is  to  examine  the  effects  of  the  total 
systems  life  cycle  costs  when  the  study  results  are  applied.  For  the 
maximum  savings  to  be  realized,  incorporation  of  these  new  training 
concepts  must  be  planned  for  as  early  in  a systems  development  as  possible. 
The  following  graphic  depicts  the  necessity  to  plan  maintenance  and 
training  concepts  early. 


€ 

Many  of  the  life  cycle  cost  parameters  are  inherent  in  the  design, 
however,  the  figure  on  the  following  page  provides  a close  examination  of 
the  cost  "DRIVERS"  on  one  case^. 

Many  Program  Managers  will  undoubtedly  feel  that  training  and  logis- 
tic support  are  either  minor  elements  of  their  program  or  someone  else's 
primary  responsibility.  For  them  the  following  is  provided: 


QUOTE  FROM  DESIGN  TO  COST  POLICY  IN  THE  ARMY  MATERIAL 
COMMAND  GUIDE,  DEC  74  AND  INDORSED  ON  3 JUL  74  IN  A 
SECRETARY  OF  THE  ARMY  MEMORANDUM  TO  THE  CHIEF  OF  STAFF, 

US  ARMY.  SUBJECT:  DESIGN  TO  COST 

"Our  ultimate  objective  is  to  minimize  the  total  life 
cycle  cost  of  ownership  of  a weapons  system.  . . . 
potential  cost/performance  trade  offs  and  engineering 
changes  must  be  evaluated  in  terms  of  their  impact  upon 
the  overall  cost  of  ownership  of  the  system  and  appro- 
priate weight  should  be  given  to  this  factor  during 
source  selection  evaluation.  Higher  acquisition  costs 
are  acceptable  provided  that  the  additional  investment 
will  be  amortized  in  a reasonable  period  of  time  through 
lower  operating  costs.  However,  designers  should  not 
be  permitted  to  lower  hardware  acquisition  costs  to 
meet  a design-to-cost  goal  if  it  would  result  in  an 
uneconomical  increase  in  operating  and  maintenance  costs." 

The  systems  of  the  future  will  be  required  to  follow  a design  to  cost 
philosophy.  As  human  resources  are  a significant  part  of  the  life  cycle 
costs  of  a system,  to  demand  that  systems  be  designed  to  human  quality 
and  quantity  resource  constraints  from  the  very  beginning  is  not  only 
reasonable  but  necessary.  The  charts  on  figure  IV-3  show  growth  curves 
that  can  be  expected  in  developing  hardware  and  documentation'  . The 
scales  of  the  chart  are  not  the  same.  In  fact  a (output)  on  the 

documentation  chart  the  same  size  as  a £>O0n  the  hardware  chart  is  in 
fact  100  times  as  great  in  output.  The^Q  is  the  change  in  output. 
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ASSESSMENT  OF  LIFE  CYCLE  COST  "DRIVERS 
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Note  the  scale  of  the  two  curves  is  not  the  same.  A $100K  investment  in 
training  and  documentation  could  be  expected  to  provide  considerably  more 
performance  increase  in  system  effectiveness  than  the  same  investment  in 
hardware. 


Figure  IV-3 
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AC 

TheAC  is  c^an9e  cos,t-  Marginal  Cost  is  Aq!  The  marginal 
benefit  is  the  worth  of  the  change  in  output.  A fundamental  proposition 
of  economic  analysis  says  "where  resource  constraints  apply,  optimal 
results  are  obtained  when  activities  are  carried  to  levels  where  they  all 
yield  the  same  marginal  return  per  unit  of  effort  or  cost"^.  Simply 
stated,  you  put  your  bucks  into  the  system  or  activity  that  returns  more 
for  the  buck  until  returns  diminish.  That  is  when  the  ratios  of  marginal 
benefits  to  marginal  costs  are  equal  at  optimal ityv  . By  equation  that 
would  be: 

MBhdw  MBt+d 

MChdw  MCt+d 

MBhdw  = Marginal  Benefit  of  Hardware 

MBj-^p  = Marginal  Benefit  of  Training  and  Documentation 

Most  system  acquisition  programs  ar«  constrained  by  dollars,  schedule, 
and  performance.  As  money  is  invested  over  time,  the  performance  parameters 
grow.  Characteristically  the  hardware  performance  growth  curve  reaches  a 
level  where  very  little  growth  is  realized  even  with  extreme  investments. 

The  marginal  return  on  the  investment  is  very  low.  This  portion  of  the 
curve  is  usually  reached  in  the  Full  Scale  engineering  Development  Phase. 
Also,  during  this  phase  the  technical  documentation  and  the  training  aids 
must  be  developed  and  demonstrated . The  management  approach  to  getting  the 
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greatest  life  cycle  cost  benefit  is  to  invest  the  money  in  the  portion  of 
the  program  to  provide  the  greatest  marginal  pay  off,  i.e.,  the  greatest 
incremental  return  for  the  incremental  investment.  If  improved  training 
aids  or  technical  documentation  provide  a greater  expected  LCC  savings 
with  a $ 1 M investment  than  the  same  $1 M investment  in  hardware  reliability 
or  maintainability  improvement,  then  the  investment  should  go  into  training 
aids  and  documentation. 

Using  a Life  Cycle  Cost  model  accessible  by  computer,  this  writer 

used  the  results  of  some  of  the  earlier  studies  to  determine  expected 

savings.  The  model  was  exercised  using  a hypothetical  but  realistic 

electronic  system  under  development.  The  model  description,  inputs  and 

outputs  are  shown  at  Appendix  A.  All  inputs  were  held  constant  except 

those  treated  as  variables  for  this  examination.  They  were: 

Number  of  job  skills  (base  case  = 4) 

Probability  of  false  maintenance  diagnostic^'^  (30%) 

Training  time/cost  (20  wks/$20K  for  Job  Skill  1,  $30K  for 
Job  Skill  2) 

Retraining  cycle  (2-1/2  yr) 

Mean  time  to  repair  (MTTR)  (varied  with  task) 

The  study  results  used  were: 

a.  Reduce  MTTR  10% 

b.  Cut  job  skill  requirements  50% 

c.  Reduce  training  time  by  50% 

d.  Reduce  false  maintenance  actions  (parts  replacements) 

e.  Examine  a 5 year  period  between  training. 
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The  model  was  exercised  after  changing  each  variable.  The  results  are 
summarized  in  Table  IV-1 . The  computer  printouts  are  at  Appendix  A.  It 
is  recognized  that  this  exercise  has  been  limited  to  a single  model, 
however,  the  manpower  and  Life  Cycle  Cost  reductions  are  so  significant 
the  results  demand  we  can  no  longer  afford  to  ignore  the  personnel  sub- 
system. 

The  results  indicate  that  c 1C?'  reduction  in  mean  time  to  repair  (MTTR): 
reduced  the  test  equipment  cost  by  10% 
reduced  the  manpower  cost  by  10% 
reduced  the  training  costs  by  50% 
reduced  the  Total  Life  Cycle  Costs  by  15.9% 

A 50%  reduction  in  the  job  skills  further: 

reduced  manpower  costs  an  additional  80% 
reduced  training  costs  an  additional  30% 
reduced  the  Total  Life  Cycle  Costs  an  additional  36.5% 

A lowering  of  the  false  part  removal  from  30%  to  10%  and  reducing 
training  time  by  50%  further: 

reduced  training  costs  an  additional  50% 
reduced  Life  Cycle  Costs  an  additional  6.3% 

With  all  other  parameters  held  steady,  the  MTTR  was  varied  from  90%  to 
80%  to  70%  resulting  in: 

reductions  in  test  equipment  cost  by  11.27%  and  12.7%  respectively 
reductions  in  manpower  costs  by  10.7%  and  12.5%  respectively 
reductions  of  training  costs  by  10.7%  and  10.5%  respectively 
reductions  of  Life  Cycle  Cost  by  5.34%  and  5.8%  respectively 
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Analysis  of  System  Cost  Parameters  as  Affected 
by  Maintenance  Training  Variations 


If  all  the  benefits  could  be  achieved  as  expected  by  the  study  results, 

the  total  life  cycle  costs  could  be  reduced  by  approximately  50%  (70,183 

to  35,133).  If  only  half  the  resulting  benefit  is  realized,  the  results 

would  still  be  a savings  of  25%  over  the  life  of  the  system.  , 

In  addition  to  the  cost  savings  available  by  improving  training  and 

technical  documentation,  other  benefits  can  be  postulated: 

o job  enrichment,  hence  higher  expected  reenlistment  rate 

o fewer  errors  in  the  contractor  produced  documentation 

o more  utilization  of  men  now  in  supervisory  positions  (presently 
71%  of  inexperienced  labor  observes  and  assists)  (2:48) 

o fewer  personnel  moves  as  average  stay  on  station  for  first  termer 
is  longer 

o technicians  leaving  service  should  be  better  trained  for  production 
in  civilian  economy  (2:47) 

o improve  reliability  and  operational  readiness  time 
o reduce  unscheduled  maintenance  manpower 

Although  a cost  savings  can  be  shown,  the  new  techniques  will  cer- 
tainly meet  with  opposition.  One  objection  to  acquiring  documents  with 
the  new  concepts  has  been  that  they  cost  more  than  conventional  documenta- 
tion and  that  project  managers  faced  wilh  competing  requirements  resist 
their  adoption.  If  the  cost  savings  potential  is  even  a small  fraction 
of  what  is  claimed  by  proponents,  the  initial  cost  should  not  be  an  over- 
riding factor.  A budget  quotation  submitted  to  AFLC  in  1971  for  completion 
of  flight  line  JPAs  for  the  C-141  was  $1,300,000  with  troubleshooting  aids 
and  $800,000  without.  McDonnell -Douglas  reportedly  estimated  that  JPAs 
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for  the  F-15  would  cost  $45  million  versus  $35  million  for  conventional 
documentation^'5®) . 

Most  estimates  indicate  the  cost  at  100%  to  125%  of  conventional 

documentation.  In  at  least  one  case,  cost  estimates  were  less  than  the 

(2*501 

estimates  for  conventional  manuals'  * . 

If  new  documents  or  JPAs  were  widely  adopted,  the  production  costs 
would  probably  come  down  due  to  the  contractor  learning  and  production 
techniques.  Current  JPA  estimates  from  contractors  accustomed  to  pro- 
ducing conventional  manuals  are  probably  inflated  because  of  uncertainty. 
The  industrial  base  for  this  kind  of  Droduct  would  expand  although  for- 
tunately there  are  currently  at  least  a half  dozen  contractors  who  have 
demonstrated  capability  in  this  area. 


SECTION  V 


CONCLUSIONS 

The  user  is  the  program  manager's  customer.  The  whole  purpose  of 
system  development  or  acquisition  is  to  satisfy  the  user's  need. 

The' service  users  have  emphasized  the  need  for  technical  and  training 
documentation  written  for  a lower  reading  level.  The  many  studies  con- 
ducted over  the  past  two  decades  provide  a means  of  satisfying  this  need 
and  at  the  same  time,  reduce  the  manpower  burden  while  improving  the  over- 
all systems  effectiveness,  figure  V- 1 , which  inherently  provides  lower  life 
cycle  cost  (the  current  support  costs  are  50-U5*  of  the  total  DoD  budget). 

“any  decisions  that  affect  the  total  life  cycle  cost  of  a system  must 
be  made  while  there  are  many  unknowns  in  system  design  and  manpower  support 

requirements.  Unfortunately  it  is  in  the  early  phases  (conceptual  and 

. »* 
v" 

validation)  of  a program  that  manpower  and  training  considerations  must  be 

v 

weighed  in  selecting  design  and  system  alternatives.  To  delay  the  con- 
sideration of  training  and  'logistic  support  until  full  scale  development 
will  have  drastic  effects  on  life  cycle  costs  as  well  as  threatening  the 
systems  effectiveness  due  to  a shortage  of  quality  personnel.  Since  the 
user  is  the  customer,  he  must  participate  in  making  trade-off  decisions  or 
provide  the  systems  effectiveness  parameters  to  enable  the  contractor  to 
recommend  trade.-offs. 
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SYSTEM  EFFECTIVENESS 


MISSION  DEMANDS 


A major  problem  is  expected  in  getting  this  philosophy  accepted  and 
the  training  and  technical  documentation  changes  made.  The  institutional 
procedures  don't  change  quickly.  As  long  as  Project/Program  Managers 
emphasize  Decision  to  Unit  Production  Cost  at  the  expense  of  Life  Cycle 
Cost  the  'hardware  fascination'  will  continue.  The  effect  of  the  personnel 
subsystem,  the  personnel  performing  operations  and  maintenance,  must  be 
considered  as  the  imperfect  subsystem  it  is.  Hardware  design  parameters 
should  be  modified  upward  accordingly  to  allow  for  the  degradation  due  to 
imperfect  personnel  and  yet  meet  the  operational  requirements  as  stated 
in  the  requirements  document. 

Life  Cycle  Costs  are  being  emphasized  in  DoD  5000.28  and  other  DoD 

instructions,  however,  most  implementers  have  difficulty  getting  a handle 

on  the  operations  and  support  costs.  The  use  of  models  such  as  the  one 

(91 

used  to  support  this  study  are  available  in  each  service'  . They  can 
provide  sufficient  data  to  support  trade-off  decisions  in  design,  support 
equipment,  manpower,  and  other  resource  limiting  areas.  To  successfully 
use  such  models,  information  such  as  maintenance  concepts,  required  opera- 
tional and  maintenance  crews,  types  of  maintenance  tasks  and  the  operational 
and  logistic  mission  profile  must  be  developed  as  inputs.  In  so  doing  the 
models  will  aid  in  design  decisions ^ ^ . The  Life  Cycle  Costs  developed  by 
this  analysis  indicates  that  although  absolute  costs  may  not  be  available, 
relative  merit  can  be  determined  by  examining  trade-offs,  and  forcing  the 
contractor  to  concentrate  on  the  system  cost  drivers. 
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It  is  also  concluded  that  the  improvement  of  training  and  technical 
documentation  may  be  a totally  new  approach  to  cost  savings  as  a viable 
alternative  for  satisfying  new  or  upgraded  requirements.  Although  a 
slight  increase  in  development  and  procurement  cost  may  be  incurred  by 
improving  the  documentation,  the  total  life  cycle  costs  should  be  reduced 
i significantly  (figure  V-2)^^. 
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Fiqure  V-2 


RECOMMENDATIONS 


RECOMMEND: 

(1)  That  DoD  direct  each  service  to  examine  the  possible  life  cycle 
cost  savings,  to  each  system  undergoing  Research  and  Development,  as  a 
result  of  using  the  documentation  and  training  concepts  developed  by  refer- 
enced studies  and  to  implement  those  showing  sufficient  cost  savings. 

Realizing  that  documentation,  training  aids,  and  devices  must  be 
developed  as  components  of  each  system,  this  writer  recommends: 

(2)  Invest  the  Research  and  Development  dollars  in  the  areas  where 
the  expected  marginal  return  is  greatest.  This  requires  acknowledging 
that  the  investment  may  legitimately  be  applied  to  improving  a subsystem 
other  than  the  hardware. 

(3)  That  each  new  requirement  document  be  examined  for  satisfaction 
of  its  requirement  by  the  following  priority  methods: 

o New  Job  Performance  Aids  on  existing  equipment 
o Product  Improvement  to  existing  equipment 
o New  System  development 

(4)  That  DoD  schools  dealing  with  systems  acquisition  take  a more 
responsible  role  in  teaching  the  development  and  contracting  for  improve- 
ment of  high  cost  subsystems  (other  than  hardware)  such  as  personnel  and 
logistic  support.  Also  that  the  DoD  schools  be  charged  with  assessing 
recommended  methods  to  obtain  Life  Cycle  Cost  Savings  and  distribute 
promising  methods  to  PM  offices  in  each  service. 


r 


IMPLICATIONS 

The  implication  of  this  study  is  that  DoD  efforts  to  improve  Life 
Cycle  Costs  is  by  emphasizing  hardware  cost  improvement  (i.e..  Design  to 
Unit  Production  Cost).  This  emphasis  overlooks  a very  fruitful  area  that 
should  provide  higher  payoffs.  Any  efforts  to  reduce  Life  Cycle  Costs 
must  be  by  early  consideration  of  the  improved  concepts  of  operations  and 
support  and  should  be  evaluated  in  view  of  the  conclusions  and  recommen- 
dations of  this  study  project. 


40 


BIBLIOGRAPHY 


1.  Shriver,  Edgar  L.  Dr.,  Study  and  Proposal  for  the  Improvement  of 
Military  Technical  Information  Transfer  Methods,  Army  HEL  Study, 

Dec  75. 

2.  Rowan,  Thomas  C.,  Improving  POD  Maintenance  Through  Better 
Performance  Aids,  DOD  Study,  Mar  73. 

3.  "Disintegrated  Logistic  Support,"  a handout  by  AF  LTC  Joseoh 
Arcieri  at  Defense  Systems  Management  College,  Program  Managers 
Course,  SELM,  Apr  77. 

4.  LTC  Buchanan,  Australian  Army;  Maj  R.  Knutson,  Utah  State  N.G., 

"The  Army  Departs  from  Training  Tradition,"  Defense  Management 
Journal , Jan  77. 

5.  "Assessment  of  Life  Cycle  Cost  Drivers,"  extracted  from  a Dresenta- 
tion  made  by  AF  LTC  Joseph  Arcieri,  Defense  Systems  Mangement 
College,  Project  Managers  Course,  Apr  77. 

6.  Herzberg,  Fredrick  L.  "Orthodox  Job  Enrichment;  A Common  Sense 
Approach  to  People  at  Work,"  Defense  Management  Journal , Apr  77. 

7.  Navy  Post  Graduate  School,  Defense  Resource  Management  Course, 

DRMEL  Handout  III  - 33,  Feb  76. 

8.  Zock,  Richard,  "Resource  Management,  Economic  Analysis  and 
Discounting  in  the  DOD,"  Air  University  Review  Sep  74. 

9.  Dover,  Lawrence  E.,  A Summary  and  Analysis  of  Selected  Life  Cycle 
Costing  Techniques  and  Models,  AFIT,  WPAFB,  Ohio,  Auq  74. 

(AD  787-183) 

10.  Arberg,  Robert  C.,  The  Impact  of  Manpower  and  Training  on  Systems 
Acquisition  Wit! l ii n the  Navy,  ISP,  DMSC,  PM  Course,  May  76. 

11.  Col  Eaton,  from  Vugraph  presentation  entitled  "Acquisition  Logistics" 
available  in  office  of  ASD  I&L,  Apr  77. 

12.  Human  Resources  Research  Organization,  Guidebook  for  the  Development 
of  Army  Training  Literature,  U.  S.  Army  Research  Institute  Study, 

Nov  75. 


41 


OTHER  REFERENCES 


U.S.  Army  Training  and  Doctrine  Command,  TRADOC  Regulation  702-1, 
Combat  Development  Program  for  Reliability,  Availability  and 
Maintainabil ity  (RAM) ,1976. 

Kidd,  Admiral  Isacc  C.  Jr.,  Getting  the  Most  Fleet  Readiness  and 
Reliability  Out  of  the  Navy's  Material  Budget,  paoer  presented 
at  an  AFMA/NSIA  Symposium  on  cost,  Washington,  D.C.  17  Aug  72. 

Statistical  Abstract  of  the  United  States,  Department  of  Commerce, 
97th  Edition,  1976. 

The  Military  Occupational  Speciality  Training  Cost  Handbook  (MOSB). 
Office  of  Comptroller  of  the  Army,  May  74. 

Department  of  Defense,  Annual  Defense  Department  Report,  FY  1976 
and  1977,  Report  of  the  Secretary  of  Defense  James  R.  Schlesinger 
to  Congress,  Feb  75. 

l/JQD  5000.28,  Design  to  Cost,  May  75. 

Roberson,  C.F.,  Useful  Life  Cycle  Cost  Estimates  for  Defense 
Systems  - an  Evaluation,  ISP,  DSMS,  Nov  75. 

DODD  5000.1,  Acquisition  of  Major  Defense  Systems,  Dec  75. 

Navy  Air  Systems  Command  Study;  Post,  Theodore  J.,  Advanced  Manpower 
Concepts  from  Sea-Based  Aviation  Systems  (AMSAS),  Dec  70. 

Ostrominski,  John,  Initiatives  for  Reducing  Personnel  and  Main- 
tenance Cost  with  Job  Performance  Aids.  Maxwell  AFB,  Ala,  Feb  72. 

Samuelson,  Paul  A.,  Economics  15th  Ed.  pg  462,  494. 

Katz,  Irving  and  Cavender,  CPT  Raymond  E.  Weapons  System  Life  Cycle 
Costing,  Operations  Analysis  Report  II,  HQ,  Air  Force  Logistics 
Command,  Jul  71. 

Army  Regulation  1000-2  Operating  Policies  for  Systems  Acquisition 
by  the  Department  of  the  Army,  Jan  777 


42 


14.  O'Donohue,  Robert  E.  Jr.,  "Life  Cycle  Costing  Beyond  the  Buzz  Word" 
Commanders  Digest,  Oct  75. 

15.  Technical  Report  NAUTRAEQUIPCEN  IH-255,  New  Concepts  in  Maintenance 
Trainers  and  Performance  Aids,  Navy  HFL,  Oct  75. 

16.  Joyce,  R.P.,  Chengoff,  A.P.,  Mulligan,  J.F.,  and  Mallory,  W.J. 

Fully  Procedural ized  Job  Performance  Aids:  Handbook  for  JPA 
Developers,  Brooks  AFB,  AFHRL,  Dec  73. 

17.  Mullen,  P.A.,  and  Joyce,  R.P.,  Demonstration  of  Fully  Procedural ized 
Job  Performance  Aids  and  Matching  Training.  Brooks  AFB,  TX,  AFHRL, 
Aug  74. 

18.  RAM  Handbook  for  the  Combat  Developer,  US  Army  Logistics  Center, 

Jul  1975. 

19.  Residori,  L.B.  Army  Major,  Contracting  for  Operational  Availability: 
An  Impossible  Goal , Study  Project  Report,  DSMC,  May  76. 

20.  DOD  and  NASA  Guide,  Incentive  Contracting  Guide,  Oct  69. 

21.  "Government  Contract  Principles,"  GAO,  Nov  70. 

22.  "Procurement  Role  in  Reliability  and  Maintainability,"  W.  J. 
Willoughby.  Reading  Article  in  Project  Management  Course,  Defense 
Systems  Management  College,  Apr  77. 

23.  Trimble,  Robert  F.,  "Can  Contract  Methodology  Improve  Product 
Reliability,"  Defense  Management  Journal , Aor  76,  pg  22-23. 

24.  Boden,  William  H.,  "Design  to  Life  Cycle  Costs,"  Special  Issue, 
Defense  Management  Journal,  Jan  76,  Vol  12,  No.  1. 

25.  "Contractor  Should  Have  Latitude  to  Make  Trade-Off  Decisions 
(Balancing  Value  Against  Cost),"  Defense  Management  Journal , Oct  73. 

26.  DOD  I 7041.3  "Economic  Analysis  and  Program  Evaluation  for  Resource 
Management,"  Oct  72. 


43 


